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ABSTRACT: An analysis of variance (ANOVA) study was performed to quantify the potential uncertainties of lift and 
pitching moment coefficient calculations from a computational fluid dynamics code, relative to an experiment, for a jet 
flap airfoil configuration. Uncertainties due to a number of factors including grid density, angle of attack and jet flap 
blowing coefficient were examined. The ANOVA software produced a numerical model of the input coefficient data, as 
functions of the selected factors, to a user-specified order (linear, 2-factor interference, quadratic, or cubic). Residuals 
between the model and actual data were also produced at each of the input conditions, and uncertainty confidence 
intervals (in the form of Least Significant Differences or LSD) for experimental, computational, and combined 
experimental / computational data sets were computed. The LSD bars indicate the smallest resolvable differences in the 
functional values (lift or pitching moment coefficient) attributable solely to changes in independent variable, given just 
the input data points from selected data sets. The software also provided a collection of diagnostics which evaluate the 
suitability of the input data set for use within the ANOVA process, and which examine the behavior of the resultant 
data, possibly suggesting transformations which should be applied to the data to reduce the LSD. The results illustrate 
some of the key features of, and results from, the uncertainty analysis studies, including the use of both numerical 
(continuous) and categorical (discrete) factors, the effects of the number and range of the input data points, and the 
effects of the number of factors considered simultaneously. 



1. Introduction 

The application of a Computational Fluid Dynamics 
(CFD) tool to a jet flap control effector on an elliptical 
airfoil-section wing has been investigated [1]. The 
Tetrahedral Unstmctured Software System (TetrUSS) [2- 
3] was used in this investigation. The system was 
developed at NASA Langley Research Center and 
includes the Reynolds-averaged Navier-Stokes flow 
solver code USM3D [4], The CFD-based jet flap 
simulations were compared to experimental results from a 
wind-tunnel test that was conducted in NASA Langley 
Research Center’s Transonic Dynamics Tunnel [5]. An 
analysis of Variance (ANOVA) study was then conducted 
using the Design-Expert software (version 6.0.10) from 
Stat-Ease, Inc. [6-10] in order to quantify the uncertainty 
of the computational results relative to the experimental 


results. Uncertainties associated with several parameters 
were analyzed. Other ANOVA studies were conducted 
and reported for other applications in Reference [11]. 

2. Experimental Description 

As shown in Figure 2.1, the model consisted of a 
modified six percent thick elliptical two-dimensional 
airfoil with 0.75% circular-arc camber and zero leading 
and trailing edge sweep. The jet flap was located at 95% 
chord and was created by exhausting a stream of high- 
pressure air from a lower surface slot that was close to the 
trailing edge (Fig. 2.1a). The jet sheet exited at 90° 
degrees to the chord axis (Fig. 2.1b). The model included 
a circular end plate of 30 in. diameter to promote two- 
dimensional flow across the wing and a splitter plate that 
offset the model 40 in. from the tunnel wall (Fig. 2.2). 
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Figure 1.2. Experimental Model. 

It was found the CFD simulation had to model the three- 
dimensional geometry of the experiment in order to obtain 
good agreement. Tests were performed at two Mach 
numbers at several different jet momentum coefficients 
(C'h). In order to be consistent with the experiment 
method, the CFD lift and pitching moment values were 
found by integrating the pressures over the wing. 

The wing was instrumented with a total of 157 static and 
total pressures taps of which 93 (47 upper & 46 lower) 
external static surface pressure taps were located at 
y/b=0.5 on the upper and lower airfoil surface. The 
sectional lift and moment coefficients were calculated by 
integrating the pressure measurements at this location. 
The model did not include a force balance. Test were 
performed at Mach=0.3 at a Reynolds of 400,000/ft and 
Mach=0.8 at a Reynolds of 900,000/ft. The momentum 
coefficients, Cju, for the jet flap data ranged from 0.006 to 
0.067. 

3. Computational Description 

TetrUSS is a software system consisting of four modules. 
It is a loosely integrated unstructured-grid CFD system 
that provides ready access to rapid higher-order analysis 
and design capability for the applied aerodynamicist. 
TetrUSS uses Gridtool [12] for geometry setup, VGRID 
[3] for grid generation, USM3D [4] for the flow solver, 
and VGPLOT [3] for post-processing. The code uses a 
cell centered, upwind biased, finite volume and 
implicit/explicit algorithm to solve the compressible Euler 
and Navier-Stokes equations on an unstructured 
tetrahedral mesh [4,13-14]. All the results obtained for 
this CFD simulation were performed using the Spalart- 
Allmaras [15] turbulence model. 

All the computations were performed using the NASA 
Ames Columbia supercomputer system. Typically, from 
7 to 28 wall clock hours were needed for each run, 
depending on size of the grid, with 96 processors working 
as a parallel system. The required CPU usage ranged from 
660 to 2700 hours of processor time per case. 


Convergence criteria were based on lift, pitching moment, 
and drag histories as well as the residual convergence. 

Generating the grid was difficult because of the small- 
scale features imbedded in the geometry, as shown in 
Figure 3.1. The aperture and surrounding region (shown 
in Figures 3.2 and 3.3) were the biggest problem areas for 
the grid generation. Five different gird densities were 
generated and used to perform a sensitivity study in order 
to assess if asymptotic behavior was observed of the 
sectional lift coefficient, C/, as a function of a grid density 
metric. Such asymptotic behavior would indicate that 
grid independence of the solution was achieved. It was 
determined in Ref. 1 that the second-most dense grid of 
the five provided sufficient fidelity and accuracy. 

The jet flap behaves like a mechanical flap, specially, a 
gurney flap. It can be seen in Figures 3.4 and 3.5 that the 
jet flap creates an area of lower Mach numbers upstream 
of it and deflects the streamlines downward just as a small 
physical flap would. The obvious advantage of the jet 
flap is that a mechanical system is not needed. However, 
one important disadvantage of the jet flap is that it has 
decreasing control authority as dynamic pressure 
increases whereas a mechanical flap would have 
increasing authority with higher dynamic pressure. 

The blowing influences the velocities over the entire 
wing. In transonic flow at Mach=0.8, the blowing 
induces a change in the shock location on the upper 
surface of the airfoil, making for larger, nonlinear changes 
in the measured or computed lift and pitching moment 
coefficients. While there is no shock at low subsonic 
Mach numbers, a similar global effect on the flow field 
occurs at the Mach=0.3 condition. 
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Figure 3.2 Cross Section of the 3D Model 


and Grid at the Trailing Edge. 



Figure 3.3 Aperture Cross Section 


Close-Up 

4. Uncertainty Analysis 

An Analysis of Variance (ANOVA) study was conducted 
using the Design-Expert software (version 6.0.10) from 
Stat-Ease, Inc. [6-11] in order to quantify the uncertainty 
of the computational results due to grid density, angle of 
attack, and C/Ll . Uncertainty analysis was also performed 
for the experimental data for potential uncertainties due to 
Cjl and to quantify the relative uncertainties between the 
computational and experimental results. 

To perform the ANOVA, factors such as the Mach 
number, blowing coefficient, and a grid density metric, 
which potentially affect the computed or measured lift 
and pitching moment coefficient, were identified and 
input to the software. Ranges of interest for the factors 
were input to the software along with the discrete 
computed or measured values of the factors. Also input 
to the software were the computed or measured lift and 



Figure 3.4 Trailing Edge Flow Visualization 
at Mach=0.8, Alpha=6.0°, (1^=0.0000. 



Figure 3.5 Trailing Edge Flow Visualization 
At Mach=0.8, Alpha=3.0°, C^=0.0115. 


pitching moment coefficient responses associated with the 
input values of the factors. In a matter of seconds, using a 
desktop workstation, a numerical model of user-specified 
order (linear, 2-factor interference, quadratic, or cubic) 
was calculated to fit the input data, residuals between the 
model and actual data were produced at each of the input 
conditions, and uncertainties (in the form of Least 
Significant Differences or LSD) for the experimental, 
computational, and combined data sets were computed. 
The LSD computed by the software indicate the smallest 
resolvable differences in the functional values (lift and 
pitching moment coefficient) given just the input data 
points from selected data sets. The software also provides 
a collection of diagnostics which evaluate the suitability 
of the input data set for use within the ANOVA process 
and which examine the behavior of the resultant data, 
suggesting transformations which should be applied to the 
data to reduce the LSD. 
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Figures 4.1 to 4.9 illustrate some of the key features and 
results from the uncertainty analysis studies. Figure 4.1 
illustrates the uncertainty analysis for the grid sensitivity 
study, described previously, at Mach = 0.3. The curve in 
the plot represents the approximate quadratic numerical 
model, which the Design-Expert software fitted to the 
input data. Design points are shown where input data 
were provided. The lift coefficient is shown as a function 
of the grid density metric with 95% confidence intervals 
in the form of least significant difference (LSD) error bars 
added; the blue square simply indicates where on the 
curve the LSD is applicable. The LSD bars indicate the 
smallest resolvable differences in the functional values 
(lift coefficient) attributable to changes in the grid density 
metric, given just the selected input data points. In this 
case, the LSD is about 0.0050, which is divided by two 
and added or subtracted from the calculated numerical 
model value to obtain high and low 95% confidence 
bounds. This means the potential lift error due to grid 
density is about +/-0.0025. 

On the finest grid, the actual lift coefficient at this grid 
density (0.7750) is expected to be between 0.7724 and 
0.7775 with 95% confidence; there is a 5% chance that 
the actual lift on this grid could be outside of these 
bounds. The actual lift coefficient, in this case, is well 
within the 95% confidence bounds. The actual lift 
coefficient value on the selected (second-most dense) grid 
(0.7741, second from left) also lies well within the 95% 
confidence interval for that grid, where the approximate 
model value is 0.7744, and the range is between 0.7719 
and 0.7769 with 95% confidence. However, the parabolic 
shape of the approximate model does not represent the 
expected asymptotic behavior of the data, which causes 
an ambiguity in the model for this region, and also causes 
the confidence interval to be broader than might be 
expected and broader than would be desired. As shown in 
Figure 4.2, the uncertainty can be reduced on the selected 
grid, simply by ignoring the finest grid results; this 
removes ambiguity about the behavior of the approximate 
lift coefficient model as the grid density is increased 
(moving toward left on the plot), and allows for a 
prediction of the actual lift coefficient on the selected grid 
to be within +/- 0.0004 of the approximate model value 
0.7741, or that the actual lift coefficient is expected to be 
between 0.7737 and 0.7745, with 95% confidence. This 
is a reduction of the prediction uncertainty by more than a 
factor of ten, by simply removing some ambiguity in the 
way the data was presented to, and modeled by, the 
software. 

Figure 4.3 presents similar results for the grid sensitivity 
study at Mach = 0.8. In this case, a linear numerical 
model was fit to the input data, again excluding the data 
from the finest grid. The 95% confidence LSD in this 
case is about 0.0070. 


Figure 4.4 illustrates a different way to perform the grid 
sensitivity uncertainty analysis. In this example, both the 
Mach = 0.3 and Mach = 0.8 have been considered 
together, with data from the finest grid again excluded. 
The lift coefficient was described to the Design-Expert 
software as a function of the numerical (continuous) grid 
density factor and a categorical factor of Mach number, 
which could only take on the discrete values of 0.3 or 0.8. 
Since the lift behavior with grid density for both grid 
studies has the same functional form, the software can 
better resolve the potential error due to grid density. The 
maximum LSD of the combined data set is now about 
0.0016. 

Figures 4.5 and 4.6 illustrate the combined uncertainty 
analysis for angle of attack and blowing coefficient, based 
upon Mach = 0.3 cases at a = 3° and 6° angle of attack, 
and Mach = 0.8 cases for a = 3.0° and 3.1° angle of attack, 
respectively. The LSD due to angle of attack is about 
0.0096 and 0.0301, respectively, for the two groups of 
different Mach data. In the Mach = 0.3 case, data are 
provided at two values of blowing coefficient. For the 
Mach = 0.8 case, data are provided at three values of 
blowing coefficient, but these data points are not co-linear 
and thus introduce ambiguity into the linear model which 
broadens the LSD more than might be expected. But 
even only data at two alpha and two blowing coefficient 
are provided, as is the case for the Mach = 0.3 data, the 
two-factor linear model (three terms) cannot perfectly 
represent the data, which is the source of a non-zero LSD 
for this data set. The data points form a “warped” plane; 
the comers can all be connected by straight lines, but the 
slope of the lines on each end of the sheet are different. 

Figure 4.7 illustrates the uncertainty analyses for two 
similar, but different, input data sets. Both computational 
and experimental Mach = 0.3 data with blowing 
coefficient up to about 0.07 are analyzed as a function of 
the blowing coefficient, C/LL . The difference between 
computational and experimental data is treated again as a 
categorical factor, which can only take on two distinct 
values (i.e., Type = Computational or Experimental), 
whereas the blowing coefficient is treated as a numerical 
(continuous) factor. The LSD is about 0.1106. Similar 
analysis for the Mach=0.8 data (not shown), with blowing 
coefficient up to 0.0240 yielded an LSD of about 0.0747. 

The same analysis as in Figure 4.7 is repeated in Figure 
4.8 for Mach = 0.3, but with only incremental jet effects 
(Jet On - Jet Off) now considered. That is, the lift 
augmentation behavior reported in Ref. 1, Figure 18, is 
now analyzed as a function of the blowing coefficient. In 
this case, the LSD is about 0.0838, but the software has 
recommended that a power transformation be applied to 
the data. Figure 4.8 shows the untransformed data, 
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One Factor Plot 


whereas Figure 4.9 shows the transformed Mach = 0.3 
data; a constant of 0.00684186 was added to each input 
lift coefficient and the sum was raised to the 1.63 power, 
and refit in this transformed space. The LSD is now 
reduced to 0.01 13 in the transformed space, which can be 
shown to reduce the LSD to about 0.0067 in the 
untransformed space. Therefore, a significant reduction 
in lift coefficient uncertainty is found using the 
incremental jet effects, combined with a power 
transformation that was recommended by the software. 
Similar jet increment lift augmentation, using a 
transformation with constant of 0.00399084 and a power 
of 1.68 for the Mach = 0.8 data (not shown), resulted in 
an actual LSD of about 0.0135, which again implies there 
is more uncertainty sensitivity due to the nonlinear effect 
of shock movement with changes in blowing coefficient. 
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Figure 4.1 Uncertainty Analysis, Grid Sensitivity 
Study at Mach = 0.3, Alpha = 6.0°. 
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Figure 4.2 Uncertainty Analysis, Grid Sensitivity 
Study at Mach = 0.3, Alpha = 6.0°, 
finest grid data excluded. 
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Figure 4.3 Uncertainty Analysis, Grid Sensitivity 
Study at Mach = 0.8, Alpha = 3.0°, 
finest grid data excluded. 
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Figure 4.4 Uncertainty Analysis, Grid Sensitivity 
Study at Mach = 0.3 and Mach = 0.8, 
finest grid data excluded. 


Figure 4.6 Uncertainty Analysis, Angle of Attack 
Sensitivity Study at Mach = 0.8, Alpha = 3.0°. 
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Figure 4.5 Uncertainty Analysis, Angle of Attack 
Sensitivity Study at Mach = 0.3, Alpha = 6.0°. 


Figure 4.7 Uncertainty Analysis, Blowing Coefficient 
Sensitivity Study at 
Mach = 0.3, Alpha = 6.0°. 
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Figure 4.8 Uncertainty Analysis, Blowing Coefficient 
Sensitivity Study at Mach = 0.3, Alpha = 6.0°, 
repeated experimental data included, no 
transformation. 
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Figure 4.9 Uncertainty Analysis, Blowing Coefficient 
Sensitivity Study at Mach = 0.3, Alpha=6.0°, repeated 
experimental data included, power transformation 
applied, k = 0.00684186, X = 1.63. 

5. Conclusions 

An analysis of variance (ANOVA) study was performed 
to quantify the potential uncertainties of lift and pitching 
moment coefficient calculations from a computational 
fluid dynamics code, relative to an experiment for a jet 
flap airfoil configuration at Mach = 0.3 and 0.8. 


Uncertainties due to a number of physical factors 
including grid density, angle of attack and jet flap 
blowing coefficient were examined. Experimental, 
computational, and combined sets of experimental/ 
computational data were analyzed. The use of both 
numerical (continuous) and categorical (discrete) factors, 
the effects of the number and range of the input data 
points, and the effects of the number of factors considered 
simultaneously, were all examined. Uncertainty bounds 
or confidence intervals (in the form of Least Significant 
Differences or LSD) were presented which represent the 
smallest resolvable differences in the functional values 
attributable solely to changes in independent variable, 
given just the input data points from selected data sets. In 
general, the data at Mach = 0.8 exhibited greater levels of 
uncertainty than did the Mach = 0.3 data for similar 
uncertainty quantification studies. The uncertainty 
bounds were also shown to highly dependent upon the 
specific characteristics of the data set(s) analyzed 
including the effects of purposely repeated and otherwise 
indistinguishable points. Diagnostics were used to 
evaluate the suitability of the input data sets for use within 
the ANOVA process, to examine the behavior of the 
resultant data, and to suggest transformations that should 
be applied to the data to reduce the LSD. The entire 
package of capabilities was found to be extremely useful 
and easy to apply in wide variety of analyses. 
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